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Abstract — A coding scheme for the discrete memoryless broad- 
cast channel with {noiseless, noisy, generalized} feedback is 
proposed, and the associated achievable region derived. The 
scheme is based on a block-Markov strategy combining the 
Marton scheme and a lossy version of the Gray-Wyner scheme 
with side-information. In each block the transmitter sends fresh 
data and update information that allows the receivers to improve 
the channel outputs observed in the previous block. For a 
generalization of Dueck's broadcast channel our scheme achieves 
the noiseless-feedback capacity, which is strictly larger than the 
no-feedback capacity. For a generalization of Blackwell's channel 
and when the feedback is noiseless our new scheme achieves rate 
points that are outside the no-feedback capacity region. It follows 
by a simple continuity argument that for both these channels and 
when the feedback noise is sufficiently low, our scheme improves 
on the no-feedback capacity even when the feedback is noisy. 



I. Introduction 

We consider a broadcast channel (BC) with two receivers, 
where the transmitter has instantaneous access to a feedback 
signal. Popular examples of such feedback signals are; 

• the channel outputs observed at the two receivers (this 
setup is called noiseless feedback); or 

• a noisy version of these channel outputs (this setup is 
called noisy feedback). 

Here we allow for very general feedback signals, and only 
require that the time-t feedback signal is obtained by feeding 
the time-t input and the corresponding time-t outputs into a 
memoryless feedback channel. This general form of feedback 
is commonly referred to as generalized feedback U], ID, fS). 
For brevity, here we mostly omit the word generalized. It 
is easily seen that our setup includes noiseless feedback and 
noisy feedback as special cases. 

We focus on discrete memoryless broadcast channels (DM- 
BCs), namely where the input and output symbols are from 
finite alphabets and the current channel outputs depend on the 
past inputs and outputs only through the current input. Our 
interest lies in the feedback-capacity region of such DMBCs, 
i.e., in the associated set of rate tuples for which reliable 
communication is possible. 

Most previous results on DMBCs with feedback focus on 
the case of noiseless feedback. For example. El Gamal ||4l 
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proved that when the BC is physically degraded, i.e., one of 
the two outputs is obtained by processing the other output, 
then the capacity region with noiseless-feedback coincides 
with the no-feedback capacity region. In contrast, Dueck 
[|5l and Kramer ||6| described some specific examples of 
DMBCs where the noiseless-feedback capacity region exceeds 
the no-feedback capacity region. In Dueck's example, the 
noiseless-feedback capacity region is known. However, outside 
these specific examples, determining the capacity region with 
feedback for (non-physically-degraded) DMBCs is an open 
problem. In fact, even characterizing the class of DMBCs 
where feedback enlarges the capacity region seems hard. This 
is partly because even the no-feedback capacity region is 
generally unknown, and partly because a computable single- 
letter achievable region for the DMBC with feedback was 
missing hitherto. Kramer Q proposed a multi-letter achievable 
region for the DMBC with noisy or noiseless feedback. 

In this paper we propose a coding scheme for the DMBC 
with generalized feedback, and present a corresponding single- 
letter achievable region. Subsequently, we analyze two new 
examples - a generalization of Dueck's channel [O, and a 
noisy version of Blackwell's channel [7j| - where our region 
is shown to exceed the no-feedback capacity region, even in 
the presence of feedback noise. Our approach is motivated 
by Dueck's example JS], and is based on the following idea. 
The transmitter uses the feedback to identify update infor- 
mation that is useful to the receivers when decoding their in- 
tended messages, and describes this information in subsequent 
transmissions. More specifically, our scheme adopts a block- 
Markov strategy, where in each block the transmitter sends a 
combination of fresh data and compressed update information 
pertaining to the data sent in the previous block. Marlon's 
no-feedback scheme ifTOl . ifTTl is used in each block to send 
the fresh data and the update information, at rates outside the 
no-feedback capacity region. The update information sent in a 
block is essentially an efficient lossy description of the auxil- 
iary inputs in Marlon's scheme from the previous block, taking 
into account the receivers' observations and the feedback 
signal as side-information. The receivers perform backward 
decoding; starting with the last block, each receiver iteratively 
performs the following two steps: 1) it decodes its intended 
data and update information in the current block; and 2) it 
uses the update information to "improve" the channel outputs 
in the preceding block, which is processed next. This strategy 
is gainful whenever the cost of the lossy description (i.e., the 
rate needed to send the update information) is smaller than the 
increase in rate it supports (i.e., the increase in capacity of the 
"improved" channel). Intuitively, this is expected to happen 
when the descriptions required by the two receivers have a 
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large common part. 

Our scheme has some ideas in common with Lapidoth and 
Steinberg's scheme for the MAC with strictly causal state- 
information at the transmitter lfT2l . ||T3| . 

Recently, another single-letter achievable region for general 
DMBCs with feedback has been proposed I16in Comparing 
the achievable region in |fT6l to ours however seems difficult. 

The paper is organized as follows. In Section [III the 
necessary mathematical background is provided. The channel 
model is described in Section In Section IIVI Marton's 
scheme for the DMBC without feedback is reviewed in detail. 
In Section [V] a lossy version with side-information of the 
Gray-Wyner distributed source coding setup is introduced, 
and an achievable region is obtained. The main result of the 
paper is introduced in Section |Vl] where the Marton and 
the lossy Gray-Wyner schemes are combined into a feedback 
scheme for general DMBCs, and the associated achievable 
region is derived. Two new examples are discussed in IVIII 
A generalization of Dueck's DMBC, and a noisy version of 
Blackwell's DMBC Q. In both cases, the region achieved by 
the new scheme is shown to exceed the no-feedback capacity 
region, using either noiseless feedback or noisy feedback, in 
the limit of low feedback noise. 

II. Preliminaries 

A. Notations 

We broadly follow the notation in IITtI - In particular, for 

def 

any real number M > 1, we use the notation [M] = 
{!,..., [MJ }. The set of positive integers is denoted by Z+. 
Also, we use upper case symbols to denote random variables, 
e.g.. A, and lower case symbols for their realizations, e.g., a. 
The corresponding alphabets are denoted by script symbols, 
e.g.. A; and |^| is used for the cardinality of A. For n £ Z+ 
we use A" and a" to denote the random sequence Ai, . . . , An 
and its realization ai, . . . , a„. 

We think of a product set of the form [2"'^i] x [2"''^] as 
being one-to-one with [2"(''i+''2)], disregarding the associated 
integer issues throughout. This assumption does not influence 
our results, as they concern the asymptotic regime n oo. 
For e > 0, we write 6{e) to indicate a general nonnegative 
function satisfying 6{e) — > (arbitrarily slow) as e ^> 0. 

A random sequence X" is said to be Px-independent- 
identically distributed (Px-i-i-d.) if 

n 

Px'4x")=l[Px{xt) 
t=l 

for all a;". Let be two jointly distributed random 

sequences, and let Py\x be some conditional distribution. We 
say that is PY\x-independent given X" if 

n 

P^„|;,„(2/"|x")=nPy|x(yt|xO 
t=l 

for aU y" and a;" with Px^{x") > 0. 

'The conference version of 1161 has been presented in the same session at 
ISIT 2010 as the conference version of this paper, see 1141 and 1151 . 



We use the notion of typicality as defined in ifTTl . For a 
finite alphabet X, a sequence a;" G X" is said to be e-typical 
with respect to (w.r.t.) a distribution Px on X if 

\tt,Ax) ~ Px{x)\ <e-Px{x) 

for all X X, where TTa-n is the distribution over X cor- 
responding to the relative frequency of symbols in a;". The 
set of all such sequences is denoted TJ^{Px)- Similarly, for 
a law Pxi - Xk over a product alphabet x • • • x Xk, we 
denote by Tp{Pxi---Xk) the set of all fc-tuples of sequences 
(.t" G Xi,...,x''^ G X^) that are jointly e-typical w.rt. 

Pxi-Xk- 

Finally, we write Z ^ Bern(p) for a a binary random 
variable taking the values and 1 with probabilities 1 — p 
and p. 

B. Basic Lemmas 

The following three lemmas are well known, and used 
extensively in the sequel. 

Lemma 1 (Conditional TypicaUty Lemma ifTTl ). Let Pxy be 

some joint distribution. Suppose x" G T"{Px) for some e' > 
0, and is Py\x -independent given X" = x". Then for 
every e> e' : 

lim Pr((a;",r") ^7;"(Pxy)) =0. 

Lemma 2 (Covering Lemma ifTTl ). Let < e' < e, and let 

X" satisfy Pr(X" G %'{Px)) I as n 00. Also, for 
each n, let Mn G Z+ be larger than 2"'' for some r > 0, 
and let {y"(?Ti)}^f^2^ be a set of Py-i.i.d. sequences such that 
{X", {i^"(™)}m=i} mutually independent. Then, for any 
law Pxy with marginals Px and Py there exists 5{e) — > 
fl5 e — > such that 

lim Pr (Vm G [M] , (X",y"(TO)) ^ 7;"(Pxy)) = 

n— ^00 

ifr>IiX;Y) + d{e). 

Lemma 3 (Packing Lemma ifTTl ). Let e > 0, and X" be an 

arbitrary random sequence. Also, for each n, let Mn S Z+ 
be smaller than 2"'' for some r > 0, and let {^"(to)}^^]^ 
be a set of Py-i.i.d. random sequences, where each Y^im) 
is independent of X". Then, for any law Pxy with marginal 
Py there exists S{e) as e such that 

lim Pr (3m G [M] s.t. (X", Y" {m)) G T^{Pxy)) = 

ifr <I{X;Y)~5{e). 

The following is a simple multivariate generalization of the 
packing lemma. 

Lemma 4 (Multivariate Packing Lemma). Let e > 0, and for 

each n let A/i,,i, M2,„, M3_„ G Z+ satisfy Mi,„ < 2"''% for 
i G {1,2,3}. Also, let {[/"(m)}^!:^ be a set of Pu^-i.i.d. 
random vectors such that {i7"(mi), ?7^(m2), ?7|'(m3)} are 
mutually independent for any toi, 7712, TO3. Then, for any law 
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Fig. 1. The two-user discrete memoryless BC with generahzed feedback. 



PU1U2U3 with marginals {PuAi^i, there exists d{e) — >■ as 
e such that 



lim Pr 3 TOi e [M,] for i e {1, 2, 3} s.t. 

{U^irm),U^\m2),U^M) e T:{PuiU2U,) 

= 



if 

n + r2 + ra < /(C/i; C/2) + /(C/3; C/i, U2) - 5(e). (1) 

Proof outline. Let £yfe = {C^r(»), C^2"(i)> C^3"(fc)) G 
I'fC^c/iC/aC/a)}- We need to show that Pr (U.^fc ^.jfc) ^ 
under Constraint By standard typicahty/large deviation 
arguments we have that 

^ 2-n{D{I{Ui:U2)+I{U3;Ui..U2)-S{e)) 

The result follows by taking the union bound over Sijk, and 
requiring that it tends to zero. ■ 

III. Channel Model 

We consider the discrete memoryless broadcast channel with 
generalized feedback in Figure [T] The goal of the communi- 
cation is that the transmitter conveys a private Message AIi 
to a Receiver 1, a private Message M2 to a Receiver 2, and a 
common message Mq to both receivers. The three messages 
Mq, Ml, and M2 are assumed to be independent and uniformly 
distributed over the finite sets [2"-^«], [2"-^i], and [2"-^^] 
respectively, where n denotes the blocklength and i?o, R2 
are the corresponding common and private transmission rates. 

Communication takes place over a DMBC with generalized 
feedback. This channel is characterized by a quadruple of finite 
alphabets X, , 3^2 , and y, and a conditional probabiUty law 
PYiY2Y\x(yi^y2^y\^) where x (E X, yi e j/2 S 3^2, 
and y E y. Given that at time t the transmitter feeds the 
symbol Xt to the channel. Receiver 1 and Receiver 2 observe 
the channel outputs ?/i t G and 2/2. t G 3^2 respectively, and 
the transmitter observes the generalized feedback yt S y, with 
probability PY^Y2Y\xiyht,y2,t,yt\xt). 

Thanks to feedback, the transmitter can produce its time-i 
channel input Xt as a function of the Messages Mg, Mi, M2 
and of the previously observed feedback outputs = 
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Fig. 2. The two-user DMBC with noise-free feedback from both outputs. 
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Fig. 3. Example of a two-user DMBC with noisy feedback. 



for some encoding function V't"^ for t G {l,...,n}. The 
DMBC and its feedback channel are memoryless, which is 
captured by the following Markov relation for t 6 [??]: 



) —Xt~^iYi,uY2^uYt) 

where Y^-' = i;, 2, . . . , Y,^t-i), for i e {1, 2}. 

After n channel uses Receiver i decodes its intended mes- 
sages Mq and Mi for i 6 {1, 2}. Namely, Receiver i produces 
the guess: 



{Mo,uM,) = ¥:'\Yn, *e{l,2} 



(3) 



(n) 

where denotes Receiver is decoding function. 

A rate triplet (i?o, ^1, ^2) is called achievable if for every 
blocklength n there exists a set of n encoding functions 
< -ipt > and two decoding functions and such 
that the probability of decoding error, i.e., the probability that 



(Mo, Ml) ^ (Mo,i, Ml) or (Mq, M2) ^ (Mq,: 



,M2 



tends to as the blocklength n tends to infinity. The closure 
of the set of achievable rate triplets (i?o, Rii P2) is called the 
feedback capacity- region of this setup, and we denote it by 

CcenFB . 

The described generalized-feedback setup includes as spe- 
cial cases the no-feedback setup where the feedback out- 
puts are deterministic, e.g., \y\ = 1; the noiseless-feedback 
setup where the feedback output coincides with the pair of 
channel outputs, i.e., Y = (^1,^2) (see Figure |2]i; and 
the noisy-feedback setup where the feedback outputs and 
the channel inputs and outputs satisfy the Markov relation 
Xt^^{Yi^t,Y2,t)-°-Yt for all t G [n] (e.g., the setup in 
Figure |3]l. In these special cases, we denote the capacity 
regions by CnoFB, CNoiseiessFB, and CnoIsyFE, respectively. 

IV. Marton's No-Feedback Scheme 

We review Marton's achievable region with a common 
message ifTOl , ifTTI . ifTTl and the coding scheme achieving this 
region. Redescribing the scheme simplifies the description of 
our feedback scheme in Section IVI-BI 
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A. Marton 's Achievable Region 

Let 7?,Marton be the closure of the set of all nonnegative rate 
triplets (i?o , i?i , i?2 ) that for some choice of random variables 
Uq,Ui, U2 over finite alphabets Uq, Ui, U2 and some function 
f:UoxUi XU2 ^ X satisfy 

Ro + Ri <I{UoMi;Yi) (4a) 
Ro + R2<I{Uo,U2;Y2) (4b) 
Ro + Ri+R2< I{Ui-Yi\Uo) + /(t/2; Y2\Uq) 

+ minI(Uo;Y,) - I(Ui;U2\Uo) (4c) 

i 

2Ro + Ri+R2< /([/o, (7i; Yi) + /(C/q, C/2; Y2) 

-I{Ui;U2\Uo) (4d) 

where X ^ /(f/o, C^i, 

([/o,f/i, (72)^X^(^1, Fa) 
forms a Markov chain, and (yi,l2) ^ ^^1^2!-^ given X. 
Theorem 1 (From QOl, HIl). 7^Marto^ C Cnofb- 

B. Marlon's Scheme 

We describe the scheme for a DMBC 3^1, 3^2, ^Vly2|x)■ 
The scheme has parameters (Wo, Wi, W2, PuaUiU2^ -^o, ^i,p, 
^i,c, R2,p^ ^2,c, -R'l, -^2' ^' ^) where 

• Uq,Ui,U2 are auxiliary finite alphabets; 

• PU0U1U2 is a joint law over these auxiliary alphabets; 

• f : Uq xUi XU2 ^ X is a function mapping the auxiliary 
inputs into effective inputs; 

• Rq, Ri.p, R2,p, Ri.c, R2,c are nonnegative communica- 

def def 

tion rates where Ri = Ri.p+Ri.c and R2 = R2,p+R2.c', 

• R'i,R'2 are nonnegative binning rates; 

• e > is a small number; and 

• n denotes the scheme's blocklength. 

def 

1) Code Construction: Define R^ = Ro + Ri,c + R2,c- The 
code consists of a single codebook Co, of [2"^"^] codebooks 

{Ci(TOc)}Lt=i^' and of L2"^-J codebooks {C2(TOc)}Lt=i^ ■ 
Codebook Co consists of [2"^"^ J length-n codewords 

{wq (TOc)}mc=i whose entries are randomly and indepen- 
dently drawn according Pug. For i = 1,2 and rric G 
pnflcj^ Codebook Ci(mc) consists of [2"^' pj bins where 
each bin m^^p G [2"^* p] contains [2"^^] length-ri codewords 

{u"(toc, ™i,p, that are randomly drawn Pui\Uo' 
independent given Uq (ttIc)- 

Reveal all codebooks to the transmitter and codebooks Co 
and {C^{■)}^"^l^ to Receiver i e {1,2}. 

2) Encoding: The encoder parses both private messages 
Ml e [2"^i] and M2 G [2"^^] into pairs of indepen- 
dent submessages (Mi_p,Afi^c) G [2"-^i-p] x [2"-^i-<^] and 
(M2,p, M2,c) G [2"^2 f ] X [2"^2-'^], and forms the new common 
message Mc = (Mo, Mi.c, M2,c) of rate Rc. 

Now, given that Mc = rric, Mi.p ~ 'mi,p, M2,p ~ m2,p, 
the encoder makes a list of all pairs (£1, £2) such that! 

(wq (mc), Uiirric, mi^p, ^i), ^2 (mc, m2,p, £2)) 

^ T^J^2^PUqUiU2)i (5) 

^The choice of e/32 will be helpful later. Here, any e' < e suffices. 
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Fig. 4. Lossy Gray-Wyner setup with side-information. 



and chooses one pair from this list at random. We call the 
chosen pair {£l,i2)- If the list is empty, it chooses 
randomly from the set of all indices [2"^i] x [2"^2]. 

The inputs a;" are obtained from the codewords Uq(toc), 
u^{mc,mi^p,£l), U2(mc,m2,p,£2) by applying the function 
/ componentwise to these three sequences; 

Xj = /(uoj (mc), Mij(mc, mi,p, £l),U2j{'mc, m2,p, (2)) 1 

3 e W- 

3) Decoding: Given that Receiver 1 observes the sequence 
y", it forms a list of all the tuples (mc,mi,p,^i) that satisfy 

«(mc),<(mc,mi,p,£"i),yn G ^^''KPuoU^y,). (6) 

It randomly chooses a tuple (rfic, mi.p, i*!) from this list (if 
the list is empty, it randomly chooses a pair (mc,mi.p) from 
j2"flcj X |-2niii,pj-) parses rhc as (mo.i, mi.c.i, m2.c.i)- It 
finally produces ?fio,i as its guess of message Mo and ihi — 
(mi^p, mi,c,i) as its guess of Mi. 

Receiver 2 produces its guesses mo, 2 and m2 of the mes- 
sages Afo and A/2 in a similar way. 

4) Analysis: See Appendix lAl 



V. Lossy Gray-Wyner Coding with Side 
Information (LGW-SI) 

In this section we study a distributed source-coding prob- 
lem and present an achievable region for this problem. The 
associated scheme will be used as part of our construction for 
the DMBC with feedback in Section [VT] 

Our source coding problem is depicted in Figure |4] Unlike 
in classical rate-distortion problems where the decoders have 
to produce sequences that satisfy certain average per-symbol 
distortion constraints, here, we require that the sequences 
produced at the decoders are almost jointly-typical with the 
source sequence. Thus, our problem is a coordination capacity 
problem lllTl . 

The rate-distortion problem corresponding to our setup is 
a lossy version of the Gray-Wyner distributed source-coding 
problem in 1191 with additional side-information at the de- 
coders. Our achievable region directly leads to an achievable 
region for this rate-distortion problem, see ll22l . Special cases 
of this rate-distortion problem have been considered by Hee- 
gard and Berger 1201 . Tian and Diggavi 1231 . and Steinberg 
and Merhav 1241 . and the lossless counterpart by Timo et al. 

ma, Ga. 
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A. Setup and Achievable Region 

Our setup is parameterized by the tuple 
3^1 , , Vi , V2 , PXY1Y2 , Pvi \X:Pv2\x,n), where 

• A", 3^1, 3^2, Vi, V2 are discrete finite alphabets; 

• PxYiY2 is a joint probability distribution over the alpha- 
bet Xxy^x y^- 

• Pvi\x and Pv2\x conditional probability distributions 
over Vi and V2 given some random variable X <E X; 

• n is the blocklength. 

In the following let {(Xf, Fi t, I2. be an i.i.d. se- 
quence of triplets of discrete random variables, with marginal 
distribution PxYiY2- Consider a distributed source coding 
setting where a sender observes the source sequence X", 
Receiver 1 observes the side-information Y{\ and Receiver 2 
observes the side-information Y^. It is assumed that the sender 
can noiselessly send three rate-limited messages Kq, Ki, K2 
to the receivers: a common message Kq to both receivers, 
a private message Ki to Receiver 1 only, and another pri- 
vate message K2 to Receiver 2 only. More precisely, the 
encoding procedure is described by an encoding function 
A(") : X'' [2"^«] X [2"-^!] X [2"-"2], which for a sequence 
X" produces the messages {Kq,Ki,K2) = A(")(X"). Each 
Receiver i, for i G {1, 2}, produces a reconstruction sequence 
= A^"-* (Kq , Ki , Fj" ) by applying a reconstruction function 
^(") . [2"flo] X [2"«'] X -> VI' to the messages Ko and Ki 
and the side-information Fj". The goal of the communication 
is that for each i e {1,2}, the reconstruction sequence 1//' 
is jointly typical with the source sequence X" according to 
Px X P 



Vi\X- 



A rate triplet (i?o, Ri, R2) is said to be e-achievable if there 
exists a sequence of encoding and reconstruction functions 
(A("\aJ"\a^"') such that: 







as n 00, for i G {1,2}. A triplet is said to be achievable 
if it is e-achievable for all e > 0. The closure of the set of all 
achievable rate triplets is denoted T^lgw- 

Let 7?-S^w be the closure of the set of all nonnegative rate 
triplets {Ro, Ri, R2) satisfying 



Ro + Ri> IiX;Vo,Vi\Yi) 
Ro + R2> I(X;Vo,V2\Y2), 
Ro + Ri+R2> I(X;Vi\Y,,Vo) 



IiX;V2\Y2,Vo) 



+ max I{X;Vo\Yi) 

ie{l,2} 



(7a) 
(7b) 

(7c) 



for some choice of the random variable Vb such that 

(1^0,1^1,1^2)^X^(11,^2). (8) 
Theorem 2. 7^i^"5^ C 7^LGw• Furthermore, 7^j^^^ is convex. 



Proof. Inclusion 7?."^^ C 7?^lgw is established in Section lV^ 
The convexity of T^lgw is proved in Appendix Icl ■ 
Notice that the region depends on the joint conditional 
distribution PviV2\VoX only through the marginal conditional 
distributions Pvi\VqX and Pv2\VoX- 



superbin 1 



superbin 2 



i 




• •• 

• •• 



L subbin of user 1 



a subbin of user 2 

Fig. 5. Double-binning structure of codeboolc Co in our lossy Gray-Wyner 
scheme with side-information. The dots depict the codewords. 



B. Scheme 

In this section we describe a scheme achieving the region 
^LGW- scheme is similar to Heegard and Berger's scheme 
for the Wyner-Ziv setup with several, differentely informed 
receivers 1201 Theorem 2]. However, our scheme also uses the 
double-binning technique for the common codebook proposed 
in II23I . but where here the double-binning is performed in 
two different ways, one way that is relevant for Receiver 1 
and the other way relevant for Receiver 2. This is beneficial 
when the quality of the side-information at the two receivers 
is very different. 

The scheme we propose has parameters Vq, PvoViV-^\x, 
-Ro,o, ^0,1' -Ro,2, Rifi, Ri,i, Rifi, R2a, R'q, R'l, R2^ e, n, 
where 

« Vo is an auxiliary alphabet; 

• PvoViV2\x is a conditional joint probability distribution 
over Vo X Vi X V2 given some X ^ X such that 
its marginals satisfy J2va,v2 ^v„ViV2\xivo,vi,V2\x) = 
Pvi\x{vi\x) and J^vo.vi Pv„ViV2\x{vo,vi,V2\x) = 
Pv2\xiv2\x); 

. i?o,o, ^0,1, ^0,2,-^1,0,^1,1, ^2,0, ^2,2 > are nonnega- 
tive communication rates; 

• Rq, R'l, R2 > are nonnegative binning rates, where R'q 
cannot be smaller than max{i?i 0, i?2,o}; 

• e > is a small number; and 

• n is the scheme's blocklength. 

1) Codebook Generation: Generate three codebooks 
Cq,Ci,C2 independentely of each other in the following way. 

Codebook Co consists of [2"^" °] superbins, each contain- 
ing [2"^oJ length-n codewords whose entries are randomly 
and independently generated according to the law Pvb. We 
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make two partitions of the codewords in each superbin, 
see Figure |5] In the first partition the codewords of each 
superbin are assigned to [2"-'^i"J subbins, each containing 
|^2"(^o-^i,o)j codewords; in the second partition they are 
assigned to [2"^^ "J subbins, each containing [2"(^o-^2,o)j 
codewords. There are thus two different ways to refer to a 
specific codeword in Cq. When we consider the first partition, 
we denote the codewords in the fci.o G [2"'''^ "]-th subbin of 
superbin /cq.o e [2"'^'»-°] by 

K(i;fco.o,fci,o,£i.o)}Cr"'""'^; 

when we consider the second partition, we denote the code- 
words in the fc2,o G [2"''^^ "]-th subbin of superbin fco,o G 
[2"«o,o] by 

K(2;fco,o,ko,^2,o)}£=r"''"'^- 
Thus, here the first index indicates whether the last two indices 
refer to the first or the second partition of the superbins. 

For i G {1,2}, Codebook Ci consists of [2"^" '] superbins 
each containing [2"^* 'J subbins with [2"^'J codewords of 
length n, where all entries of all codewords are randomly and 
independently drawn according to For fc^.i G [2"^' '], 
we denote the codewords in the ki i-th subbin of superbin 
ko,i G [2"««-'] by 

All codebooks are revealed to the sender, and codebooks 
{Co,Ci} are revealed to Receiver i G {1,2}. 

2) LGW-SI Encoder: Given that the encoder observes 
the source sequence X" = x", it searches the codebooks 
Co,Ci,C2 for a triplet of codewords Ug (1; fco,07 ^1,0, ^i.o) G Co, 
Vi{ko,i,ki^i,£i) G Ci, ^2(^0,2,^2. 2, ^2) G C2 such that for 
i€{l,2}: 

(X",z;o"(l;fco.o,fci,o,^i.o),<(fco,,:,fcM,^0) e Trf2{PxVoV,)- 

(9) 

It then forms a list of all tuples of indices 

(fco,o,fci, 0,^1, 0,^047^=14: ^1:^0, 2,^2, 2, ^2) satisfying 
If the list is non-empty, the sender chooses one tuple 
from this list at random; otherwise, it randomly chooses 
a tuple (fco,o,fci,o,^i,o,fco4,fci4>:^i>^o,2,fc2,2,-^2) from 
the set [2"^o.o 



X [2"-^i.o] X [2"(^o--Ri,o)] X [2 
X [2"^o 



nB.o 



X [2 



nR2- 



. . * . [2'^ 
chosen indices by fco,oi ^i,0' ^i,0' ^04' ^141 ^i' ^0,21 ^'2,21 ^2- 
Also, define (fc2_0'^2,o) such that Wq (2; feg g, fcj.p, ^2.0) ^"d 
Vq{1; fcp Q, klQ,£l q) refer to the same codeword in Cq. 

The encoder then sends the product message A'o = 
(fcg oi ^0 ii '''O 2) '^o tioth receivers, the product message A'l — 
(fc*Q,fc5'i) to Receiver 1 only, and the product message 
K2 = (^2 0, ^2 2) to Receiver 2 only. 

3) LGW-SI Decoder: Receiver i G {1,2} first parses the 
common message A'o as (i^o,o, -^^04, -^^0,2) and its private 
message Ki as Kj = {Ki,o, Ki,i). Then, given that Re- 
ceiver i's side-information is Y" = yf and that Kq^q — fco.o, 
Ka,i = fco.i, Kifi = kifi, and Ki ^ = fc; ,;. Receiver i seeks a 
codeword Wq (*; ^0,0, ^4,0, ^i,o) in codebook Cq and a codeword 
^r(^o,i, ki^i,£i) in codebook Ci such that 



2 . We denote the 



If exactly one such pair of codewords exists. Receiver i pro- 
duces as its reconstruction sequence V" = vl^iko,i,ki^i,£i). 
Otherwise, it randomly chooses a triplet [k'^ ^,k[^,£^) from 



the set [2 



X [2 



nRi, 



[2"^i] and produces as its recon- 



struction sequence V-^ = vf{k'^ ,j, k[ , , ^';). 

4) Analysis: In Appendix |B] we show that under Con- 
straints (|7]i the failure probability of our scheme tends to as 
n — !> 00. The existence of a deterministic coding scheme with 
vanishing failure probability follows from standard arguments. 

VI. Main Result for DMBCs with Generalized 
Feedback 

A. Achievable Region 

Consider a DMBC with generalized feedback given by 
X ,yi,y2^y tPy-^Y2Y\x- dinner be the closcd convex hull 
of the set of all nonnegative triplets (i?o,^i,^2) that satisfy 
Inequalities (fT2l i shown on top of the next page, for some 
choice of auxiliary random variables ([/q, J7i, f72, Vq, Vi, V2) 
and function / such that X = f{Ua,Ui,U2), 



iVo,Vi,V2)^iUQ,Ui,U2, r )^( Yi , Y2 ) 



and 



{Ua,Ui,U2)^X- 



<Yi,Y2,Y) 



(10) 



(11) 



form Markov chains, and {Yi,Y2,Y) — j YiYoY\x- 

Notice that for noise-free feedback where Y = (Yi, Y2) the 
Markov chain (fTOl i is satisfied for any choice of the auxiliary 
random variables ([/q, Ui, U2, Vq, Vi, V2). 



Theorem 3. TZ\r 



CC, 



GenFB- 



k £ 



The proof of the theorem is given in Subsection IVI-BI A 
few remarks are in order: 

Remark 1. The region TZmnei: includes TZunnon, because when 
for a given choice of {Uq, 1/1,1/2), constraints (112b are spe- 
cialized to (Vq, Vi, V2) = const, then it results in the Marton 
region (|4]l. The inclusion is also clear from the construction 
of our scheme in Subsection WI-B\ ahead. 

Remark 2. In our coding scheme we can allow f to be a 
randomized function. In this case, the scheme achieves the 
region TZianei but where the input X can be an arbitrary 
random variable satisfying the Markov chain jl lb . 

We can also superposition all the codebooks on a Pq-i.i.d. 
random vector Q" that is known at the transmitter and both 
receivers. In this case, the joint typicality checks need to be 
modified accordingly. The new scheme achieves a region as in 
dinner but where the mutual information constraints (112b need 
to be conditioned on Q and the Markov chains in (llOb and 
(II lb require Q in the middle position. 

It is not clear whether these changes result in an improved 
region compared to 7?.innei-- 

Remark 3. Recall that for fixed finite alphabets, the Shannon 
information measures are continuous (say w.r.t. Euclidean 
distance) in the joint distribution h28'i . Fix the channel's input, 
output, and feedback alphabets. Then for any fixed choice 
of {PU0U1U2J, PvoViV2\UoU±U2y)' quantities on the right- 
hand side of Inequalities l[12i are continuous in PyiY2Y\x- 
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i?o + i?i < /(C/o, (7i; Fi, Fi) - /(C/o, C/i, t/2, y; 14), \Yi) (12a) 
i?o + i?2 < /(C/o, C/2; ^2, V^2) - /(C/o, C/i, C/2, y; Fo, ^21^2) (12b) 
Rq + Ri+R2< I{Ui-YuVi\Uo) + /(C/2; 1^2, 1/2|C/o) + min /(C/o; F., V^) - /(C/i; C/2IC/0) 

ie{l,2} 

-/(C/o,C/i,C/2,y;l/i|Vb,Yi)-/(C/o,C/i,C/2,r;-t^2|T^o,i^2)- max /(C/o, C/i, C/2, F; Foli^.) (12c) 

i6{l,2} 

2/?o + /?i + /?2 < /(C/iC/o; Fi, 1/1) + /(C/2, C/q; ^2, '^^2) - /(C/i; C^2|C/o) 

-/(L/o, C/i, C/2, F; Fo, |Fi) - /(C/o, C/i, C/2, F; Fo, V^2|F2) (12d) 



Remark 4. By the previous remark, the following conclu- 
sion holds for any DMBC PyiY2\x with feedback alphabet 
3^ = 3^1 X 3^2- Assume that the region Thinner associated with 
noiseless feedback (i.e., Y ~ (Fi, F2)j strictly contains CnoFB- 
Now, if we consider a noisy feedback channel Py\xYiY-2 ^^'^^ 
is close enough to the noiseless feedback (i.e., Y close to 
(Fi, F2)j, then also the region 7?.innei- associated with this noisy 
feedback strictly contains CnoFB- 



B. Scheme achieving T^innner 

Our scheme combines Marlon's no-feedback scheme of 
Section HV^ with our LGW-SI scheme of Section EH using 
a block-Markov framework. We first present the high-level 
idea of the scheme, which is also depicted in Figure |6] 
Transmission takes place over B+1 consecutive blocks, where 
the first B blocks are of length n each, and the last block is 
of length for 7 > 1. We denote the input/outgut/feedback 
sequences in Block h G [B] by X^'j^j, F"^^^ F^^^, respec- 
tively, and the input/output sequences in Block B + 1 by 
-^(B+i), F,"s+i)- The messages to be sent are in a product 
form Mi = (A/,;^(i), . . . ,/V/i^(s)), for i 6 {0,1,2}, where 
each Afj ({)■) is uniformly distributed over the set [2"^']. The 
effective rates of transmission are thus 



B 



B B B 

Roj "J^ /?1, ~ — '. /?2 



-7 



B+7 "B+j 



(13) 



and approach {Rq, Ri, R2) as the number of blocks /? — > cxd. 

In each block b the transmitter uses Marton's no-feedback 
scheme to send the Messages /V/o,(f,), A/i,(h), A/2,((,) together 
with update information /fo,(6_i), /^i (f,_i), /ir2,(h-i) pertain- 
ing to the messages sent in the previous block. An exception 
is the first (resp. last) block where only the message tuple 
(resp. update information) is sent. The update information is 
constructed in a way that when (/io.(h), /^'i.(h)) is available 
at Receiver i, the latter can use it to "improve" its block-6 
observations F/'^^^. This facilitates the decoding of the cor- 
responding messages A/o.(h), A/i (h), A/2_(b), which otherwise 
might not have been possible to decode reliably. The update 
information is generated via the LGW-code described in Sec- 
tion IV-BI The code is designed for an LGW-setup where the 
encoder's "source sequence" consists of the auxiliary Marton- 
codewords and the feedback signal, and where the receivers' 
"side-informations" consist of their respective channel outputs. 

Each Receiver i, for i G {1,2}, performs backward de- 
coding. It starts from the last block and decodes the update 



information (/\o,(_b), /i^i,(B)) based on F"s+i)- Denote its 
guess by /i^o,i,(B), /^i,(S)- Then, for each block b G [B], 
starting from block B and going backwards, it performs the 
following steps: 

1) Using (/Co.fc, /^i.t), it "improves" its block-5 outputs 

2) Based on these "improved" outputs, it then decodes 
the data (A/o_(fc), A/^ ({,)) and the update informa- 
tion (/iro,(6-i), /^i,(b-i))- We denote the corresponding 
guesses by (Mo,(6), Mi,(6)) and (A'o,»,(h-i)' ■^i,(f'-i))- 

We now describe the coding scheme in more detail. Our 
scheme has parameters [Uq, Ui, U2, Vo, Vi, V2, PuoU^u^^ /, 

^VoViV2\UoUiU2Y' ^0, /?1, /?2, R'l, /?2, Ru /?2, R'o, R'l, 

R'2, e, 7, n, B), where; 

• Uq, Ui, U2, Vo, Vi, and V2 are finite auxiliary alphabets; 

• PU0U1U2 is a joint probabihty law over Uq xUi x U2', 
. / is a function f : Uq x Ui x U2 ^ X\ 

• -^VoViV2\UaUiU2Y ^ Conditional probability law over 
Vo X Vi X V2 given a tuple (C/q, C/i, C/2, F); 

• Rq, Ri, R2, Rq, Ri, R2 are nonnegative communication 
rates; 

• R'l, R'2, R'q, R'l, R'2 are nonnegative binning rates; 

• e > is a small number; and 

« n, 7, and B are positive integers determining the 
scheme's blocklength. 

1) Code Construction: For each block b G [B] we construct 
a Marton code for a DMBC with parameters (A", x Vi, 3^2 x 
^2, P{YiVi){Y2V2)\x) using the code construction in Subsec- 
tion IIV-BII As parameters of this construction we choose: 

• the auxiliary alphabets Uo,Ui,U2', 

• the joint law Puo,Ui,U2 o^^r these alphabets; 
> the function f : Uq x Ui x U2 X; 

m the nonnegative communication rates Rq, Ri,p, R2^p, 
Ri,c, R2,c where we require that Rq ^ Rq + Rq, 
Ri^p + /?i,c = i?i + Ri, and /?2,p + /?2,c = R2 + R2\ 

• the nonnegative binning rates R'l, R'2, 

• the small number e; and 

• the blocklength n. 

For block /? + 1, we use a Marton scheme for the DMBC 
with parameters (A", 3^i, 3^2, /i7y,|x) of block length 771 
where the scheme is chosen as to achieve the rate triplet 
(7^^/?o, 7^^/?i, 7^^/?2)- To make sure that such a scheme 
exists, we assume throughout the proof that the single-user 
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t = n 



t= Bn + 7n 



fresh data 

Mo,(i),Afi,(i,,M2,(i, 


fresh data 

Mo.(2),Afi,(2),Af2,(2) 


fresh data 

Afo,(3),Mi,(3),M2,(3) 




update info. 

-^0,(1)1 -^2,(1) 


update info. 

Ko,{2), Ki ^2), -^2,(2) 



fresh data 




update info. 

-f , (-B-1 )i -f 1 , (-B-1 )i -'f 2 , (B-1 ) 


update info. 



Block 1 Block 2 Block 3 

Fig. 6. Block-Mai'kov strategy of our feedback-scheme. 



Block B 



Block B + 1 



channels Pyi\x /V,|x both have positive capacities]^ 
Under this assumption, it is readily verified that for 7 > 1 
large enough such a scheme exists. 

In what follows, let (/?({,), *i'i.(h), ^2,{b) denote the encoding 
and decoding rules corresponding to the Marton-code in block 
6, for any b G [B+1]. Also, let the triplet (t/o"(6)' ^"(6)' f^2!(fc)) 
denote the auxiliary codewords produced by the block-6 Mar- 
ton encoder for any b € [B], and let -''^(J,) , ^^'(f,) , i^2"(h) 
and Yfj^^ denote the corresponding blocks of channel in- 
puts/outputs/feedback outputs. 

Then, consider the LGW-SI setup with the following pa- 
rameters; 

• the source alphabet {Uq xUi XU2 x y)', 

• the decoder side-information alphabets J^i and 3^2; 

• the reconstruction alphabets Vi and V2; 

• the source-side-information law P(^UoUiU2Y)YiY2' 

. the reconstruction laws P^^^ |uo, wi, U2, y) = 

^vo,V2 PvoV,V2\UoU,U2Yi'"0, Vi,V2\uo, Ui, U2, v) 

and Py^^^^jj^ij^Yi^^\uo,ui,u2,y) 

T.vo,v^PvoV,V2\UoU^U2Yi^O,Vi,V2\uo,Ui,U2,y). 

For this LGW-SI setup we construct for each block b G [B] 
an LGW-SI code as described in Subsection IV-Bll Our con- 
struction has the following parameters: 

• the auxiliary alphabet Vq; 

. the conditional law Pv,v^V2\UoU^U2Y' 

• the nonnegative rates Rq^, Ro,i, Ro,2, Ri,o^ Ri,i, R2.0, 
^2,2, R'o, R'l, R'2'^^ 

• the binning rates R^,R'i,R2 > 0, where i?g cannot be 
smaller than max{i?i_o, ^2,0}; 

• the small number e/2; and 

• the blocklength n. 

In what follows, let A(f,), Ai (h), and A2,(6) denote the LGW-SI 
encoding and decoding rules corresponding to these codes. 

2) Encoding: In the first block 6 = 1, the transmitter 
forms the product messages Jo.(i) = (-^'^o.(i)i 1)^ •^1,(1) '= 

def 

(Ml (1), 1), and J2,(i) = (^^2,(1)1 1)' and applies the Marton 
encoding rule to this triplet Jo,{i), •^2,(1)- 

In blocks b G 2, . . . ,B the transmitter first applies 
the LGW-SI encoding function ^{b-i} to its "source se- 
quence" (C/o;(fc_i),C/"(b_i),C/2"(h-i)'^(h-i)) to generate the 
update messages {Kq^(^i,-i), Ki^^b-i), K2,(b-i))- (Recall that 
C^o (6-1)' ^"(6-1)' ^2* (6-1) denote the Marton auxiliary code- 
words produced in the previous encoding step.) The transmitter 

def 

then generates the messages J,;,(f,) = (Mj (j), i^.j and 

^^Wlien one of the two single-user channels has capacity 0, then the 
broadcast problem is not very interesting. In fact, in this case both the capacity 
regions with noiseless feedback and with no-feedback are degenerate. 



encodes them via the Marton encoding rule ^^(b)- It finally 
sends the outcome of this encoding over the channel. 

In the last block B + 1, the transmitter first applies 
the LGW-SI encoding function X(^b) to the sequences 

(^o!(s)'^r,(B)'^2^(B)'^('B)) t° generate the update mes- 
sages Ki i^B), for i G {0,1,2}. It then forms the tu- 

def def 

pie JoXB+i) = (l,-f^'o,(i3)), -^i,(s+i) = (1,^U.(B)), and 

def 

■^2,(3+1) = (l5-^2.(B)) and encodes them via the Marton 
encoding rule ip(^B+i)- 

3) Decoding at Receiver i: Decoding is performed back- 
wards, starting from the last block. Receiver i first applies 
the decoding rule ^i,(B+i) to the outputs ^j"^^!) attempting 
to decode the indices {Jo,{b+i), Ji.{B+i))^ and parses its 
guess {Jo,i^{B+i),Ji,iB+i)) as Jo,i,(b+i) = {^,Ko,i.{B)) and 

Ji,(B+l) = (l,^i,(B))- 

Now, for every block b G {1, . . . ,B}, starting with block 
B and going backwards, the receiver performs the following 
steps. It applies the LGW-SI decoder A; (b) to its guess of the 
update messages {K^ i ^p-^^Ki^^p-^) obtained in block 6+1, and 
to its "side-information" Yl^^p-^ ■ It then applies Marton's decod- 
ing rule to the pair ^""(6)), where V^^^-^ denotes 
the reconstruction sequence produced by the LGW-SI decoder 
Ai ({,)■ Finally, it parses the guess produced by Marton's de- 
coding rule (Jq,j,(6), ii,(6)) as Jo,i,(6) = (Mo,i,(6), A'o,i,((,-i)) 
and Ji^^b) = (Mi^(i,),K,^(i,^i)). 

Receiver i's guess of the messages A/q and Mi are 
the products Mo,i = (Mo,,;^(i), . . . , Mo_i^(s)) and Mi = 

(7\4(i),...,A4(B)). 

4) Analysis: In Appendix |D] we show that under Con- 
straints (fT2] i the error probability of our scheme tends to as 
n ^ 00. The existence of a deterministic coding scheme with 
vanishing error probability follows from standard arguments. 

VII. Examples 

A. The Generalized Dueck DMBC 

In ||5] Dueck presented the first example of a DMBC 
where noise-free feedback increases capacity. In his setup, the 
channel input consists of three bits, X ~ {Xq,Xi,X2), and 
each of the two outputs of two bits, Yi = (^t, 1,^1,0) and 
^2 = (^2, 0,^2, 2) where 

^1,0 = ^2,0 = Xi), 

yi,i = Xi ® z, 

>2,2 ^X2®Z. 

Here, the noise Z is Bern(l/2) and independent of the inputs, 
and © denotes addition modulo 2. 
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Mo 
Ml 
Mo 



Xo 



Receiver 1 



X2 



>( 



■4- 

Z2 



Receiver 2 



Mo,i 
Ml 



A/0,2 
M2 



Fig. 7. Generalization of Dueclc's DMBC witli feedback example. 

Obviously, without feedback, the outputs Yi i and Y2.2 are 
useless. Thus, the no-feedback-capacity is given by the set of 
all nonnegative rate triplets {Ro,Ri,R2) satisfying 

Ro + Ri+R2< 1. 

With noiseless feedback, the capacity is increased. 

Theorem 4 (Dueck Q). The noiseless feedback capacity of 
Dueck's DMBC is given by the set of all nonnegative rate 
triplets {Rq, Ri, R2) satisfying 



Rq + Ri<1 



and 



Ro + R2< 1. 



(14) 



Proof. The converse follows from the cutset bound. The 
achievability by the following simple blocklength-(n +1) 
scheme. The transmitter sends lossless descriptions of the Mes- 
sage pairs {Mq, Mi) and {Mq, M2) using the inputs 
and {X2.t}"^i, respectively. Additionally, for t ~ 2, . . . , {n + 
1), it repeats the previous noise symbol as Xo^t = Zt^i- The 
transmitter knows Zt-i at time t because it is cognizant of 
the input Xi,t-i (or X2^t-i) and, through the feedback, also 

of = + ZfLi (or r2,t-i = ^2,t-i + Zt-i). 

Notice that each Receiver i G {1,2} learns the noise 
sequence {ZfjJLj from its sequence of outputs {Yifl_t\t=2 ■ 
Receiver i can thus compute the channel inputs t = 
Yi^i,t — Zt, for t = 1, . . . , ri, and recover the desired pair of 
messages (A/q, A/^) whenever the sum-rate Ro + Ri is smaller 
than Letting the block-length n tend to infinity, we get 
the desired achievability result. ■ 

We generalize Dueck's setup to the DMBC depicted in 
Figure [T] We assume that all three binary channels are noisy, 
and the first and third channels are corrupted by different 
noises. Thus, as before, the channel input consists of three 
bits, X = {Xi, Xq, X2), and each output of two bits, Yi ~ 
(^1, 1,^1,0) and Y2 = (>^2, 0,^2,2)- However, now, 

Yi.o 



l2,0 


= Xoe 


BZa 




= Xie 


BZi 


^2,2 


= X2e 


BZ2 



where Z^, Zi, Z2 are binary random variables of a given joint 
law PzoZiZ2- 

Proposition 1. The no-feedback capacity region of the gener- 
alized Dueck DMBC is the set of all nonnegative rate triplets 
{Roi Ri, R2) that satisfy 

Ro + Ri<2-HiZo,Zi), (15a) 
Ro + R2<2-H{Zo,Z2), (15b) 
Ra + Ri+R2<3- HiZa, Zi) - H{Zo, Z2). (15c) 



Proof. The no-feedback capacity of a DMBC depends on the 
channel law ^ViY,|x(2/1j 2/2|a;) only through the marginal laws 
PYi\x{.yi\x) and PY2\x{y2\x) (see e.g., |23). We therefore 
assume in the following that Z2^^ZQ^y—Zi. The con- 
verse follows then simply by applying the cutset bound to 
this modified setup. The achievability follows from Marton's 
achievable region. More precisely, if in the region in (|4]i we 
choose Uq, 1/1,1/2 to be i.i.d. Bern(l/2) and Xi = Ui, for 
i e {0, 1, 2}, then it evaluates to our region in ( fTSl ). (Notice 
that since we choose Uo,Ui,U2 independent, constraint ( l4dl ) 
on 2Rq + Ri + R2 is not active.) ■ 
Our scheme in Section IVI-BI allows us to obtain the capac- 
ity region for the Generalized Dueck DMBC with noiseless 
feedback when 

H{Zo,Zi)<l and H{Zo,Z2)<l. (16) 

Theorem 5. Under condition ( 116b and when no common 
message is sent, i.e., Rq ~ 0, the noiseless-feedback capacity 
of the Generalized Dueck DMBC is the set of all nonnegative 
rate pairs (i?i,i?2) satisfying 



Ri<2-H{Zo,Z,), 
R2<2-H{Zo,Z2), 
R1+R2 H{Zo,Zi,Z2). 



(17a) 
(17b) 
(17c) 



Proof. The converse follows from the cutset bound. The direct 
part follows from Theorem |3] by taking the convex hull of the 
achievable regions that result when (fT2] | is evaluated for the 
following two choices: {Uq, Ui, U2) i-i-d. Bern(l/2); Xi = Ui 
for i £ {0,1,2}; V, = {Xa,X,) for i G {1,2}; and either 
Vo = {Zq, Zi) or Vo = {Zq, Z2). (Notice that since Uq, Ui,U2 
are independent. Constraint ( I12db is subsumed by Constraints 
(fT2al i and ( fT2b] i.) ■ 
In view of Proposition [T] we have the following corollary 
to Theorem |5] 

Corollary 1. If the triplet {Zq, Zi, Z2) satisfies (116b and does 
not form the Markov chain Zi — Zq — Z2, then noiseless 
feedback strictly increases the capacity of our Generalized 
Dueck DMBC. 

Let's briefly consider the case of noisy feedback Y = 
(^1,1 ® Wi , ri,o © Wo, ^2,2 © W2) where (Wo, Wi, W2) are 
arbitrary distributed binary random variables, with marginals 
Wi ~ Bern((7i), for qo,qi,q2 G (0, 1). Evaluating Theorem[3] 
for this noisy-feedback setup is cumbersome and left out. But 
from Corollary [T] and the continuity considerations mentioned 
in Remark m we can conclude the following. 

Remark 5. If the noise triplet {Zq,Zi,Z2) satisfies ( 1161 ) 
and does not form the Markov chain Z\ — Zq — Z2, then 
for any sufficiently small value 0/ max{(7o, (j'2}, the noisy 
feedback introduced above enlarges the capacity region of the 
Generalized Dueck DMBC. 



B. The Noisy Blackwell DMBC 

Consider the noisy version of the Blackwell DMBC in 
Figure [8] The input alphabet is ternary X ~ {0, 1, 2} and 
both output alphabets are binary = 3^2 = {0, 1}. Let Z 
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Fig. 8. A noisy version of Blackwell's DMBC with noiseless feedback. 

Bem(p), with p < ^, be independent of X. The channel law 
PyiY2\x is described as follows. 



Z X = Q 
I- Z X = 1,2 



Z X = 0,1 
1- Z X = 2. 

(18) 



When p = 0, the described DMBC specializes to Blackwell's 
DMBC. For this case the capacity region with and without 
feedback is given by Marton's region. We consider noiseless 
feedback and present an achievable region for this setup based 
on the region 7?.inner in Theorem [3] 

Let Uo,Ui,U2 be binary random variables, where Uq ^ 
Bern(i), and where given Uq = the pair {Ui, U2) has joint 
conditional law 





f/2 =0 


U2 = l 




= 


a 





Ui 


= 1 


l-a-l3 


/3 



UiU2\Uo=0- 



for some nonnegative a, (3 satisfying a + /3 < 1, and given 
[/o = 1 it has joint conditional law 



P, 





U2=0 


U2 = 1 


Ui 


= 


fi 





Ui 


= 1 


l-a-13 


a 



UiU2\Uo = l- 



Set X = Ui + U2 (real addition), and let Vi = Ui, V2 = U2, 
and Vo = Vi®Yi= Z. Evaluating the region in ( 112b for this 
choice of random variables, we obtain the following theorem. 

Theorem 6. All nonnegative rate triplets {Rq, Ri, R2) satis- 
fying 



Rq + i?i < /if, 
i?0 + < hb 



2 

a + P 



*pj- hbip) 
★ pi - hb{p) 



Ro + Ri+R2<hb[ [ ^^-^ 



1-/3 



'^-hb ( ^ ) - hh{p) 



1-a 

a + l3 



irp - 2hb{p) 



2Ro + Ri+R2< 2hb 



+ H{[a,/3,l-a-/3])- hb{a) - hbiJ3) 

are achievable over the Noisy Blackwell DMBC. Here, 
H{[p,, . . . 1 P. log j-; hb{p) H{[p, 1 - p\); 

def 

and ^ -k p = (1 — 7)73 + 7(1 — p). 



+ 









No feedback upper bound (20) 




*. 
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Fig. 9. Bounds on tfie maximum sum-rates R\ + R2 that are achievable 
over the Noisy Blackwell DMBC with no feedback and noiseless feedback. 



Let US consider the sum-rates Ri + R2 guaranteed by the 
region above. To that end, we set i?o = and note it is 
sufficient to consider only the last two inequalities. We get 
the following corollary to Theorem |6] 

Corollary 2. With noiseless feedback, our scheme achieves 
all nonnegative rate pairs (i?i,i?2) satisfying Inequality ( 1191 ) 
shown on top of the next page. 

For comparison, let us now upper bound the sum-rates 
i?i + i?2 that are achievable without feedback. Since the no- 
feedback capacity of a DMBC depends only on the marginals 
Pyi \Xi Py2\x 1271 . the capacity region for the Noisy Blackwell 
channel remains the same if in the definitions of Yi and Y2 (see 
( fTSl )) we replace Z by independent Bern(p) random variables 
Zi and Z2, respectively. Computing the cut-set upper bound 
for this latter setting, we obtain that all rate pairs (i?i,i?2) 
that are achievable without feedback must satisfy 

Ri + Ra 

< sup {H(^[a{p ~ pf + pp,p^ + 2ap{p - p), 



ae(o,i) 



+ 2ap{p -p),a{p-p)'^ +pp])} ~ 2hb{p), 

(20) 



where p 1 



p. Figure |9] depicts the bounds (T% and (|20] | 
together with a cut-set upper bound on the sum-rates i?i + R2 
that are achievable with noiseless feedback. By this Figure |9] 

Corollary 3. Noiseless feedback enlarges the capacity region 
of the Noisy Blackwell-DMBC. 

Remark 6. Let Y = {Yi®Wi,Y2® W2), where (W^i, ^^2) 
are jointly distributed binary random variables with marginals 
Wi ^ Bern(gi), mutually independent of {X,Yi^Y2). By the 
continuity argument in Remark |?] for any p G (0, 1) and 
small enough max{(7i,q2}, noisy feedback strictly enlarges 
the capacity region of the Noisy Blackwell-DMBC with noisy 
feedback. 
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a+p<l 



2hb I I ^^4^ Up 1 + H ([a, /3, 1 - a - - - hiP) ~ 2hb{p) } (19) 
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Appendix A 
Analysis of Marton's Scheme 

We analyze the average probability of error of Marton's 
scheme averaged over the random messages, codebooks, and 
channel realizations, see also ifTOl . ifTTl . ifTTl . Recall that an 
error occurs whenever 

(Mo,i,A/i) ^ (Mo, Ml) or (Mo,2,A/2) ^ (Mq, Afa). 

By the symmetry of the code construction 

Pr [error] = Pr [error] A/^ = Mi^p = M2,p = 1]. 

To shorten notation we denote the event that Mc ~ ^h,p = 
M"2,p = 1 by X = 1. Also, let 

• £() be the event that there is no pair (^1,^2) £ [2"^i] x 
[2"^2] satisfying 

(C/^'(l),t/r(l,l,£i),(72"(l,l,4)) e X%iPuoU,u,)- 

• £oi be the event that 

{u-{i),u-{i,i,L:),Yn i T}-\Puou.Y.), 

where LJ and L2 denote the pair of indices chosen during 
the encoding step. 

• £ii be the event that there is a fhc ^ 1 such that 

(C/o"(m,),C/r(m„l,L*),r,") £ T^^HPuoU.y.) ■ 

• £21 be the event that there is a pair ^ 1 and such 
that 

• £31 be the event that there is a tuple rhc 7^ 1, mt ^ 1, 
and £i such that 

When the event {£^ n £,5 ^ n f f ^ n f| ^ n f| j) occurs, then 
Receiver i G {1,2} correctly decodes its desired messages Mq 



M = 1 



Pr (error|7W = 1) 

<Pr(fou(|JUf,,.) 

\ ^i=ij=i / 

<Pr{£o\M^l) 
2 

+ ^ ( Pr {£o^\£^, = 1) + Pr {£u\£S„M = 1) 

1=1 

+ Pr (£2.1^0*, = 1) + Pr {£3^\£o^: M = l)). 

We consider each of the terms separately. A nonnegative 
function S{e) satisfying (5(e) — > as e — > can be chosen 
such that the following statements hold. 

• By the code construction and by a conditional version of 
the covering lemma (Lemma |2]l. 



whenever 



lim Pr(£:o|Af = 1) = 0, (21) 



R[+R'2> IiUi;U2\Uo) + S{e). (22) 



Since the channel outputs Y" is a Py. |x-i-i-d. sequence 
given X" and by the conditional typicality lemma 
(Lemma [B, 



lim Pr(5oi|^^0'-^ = 1) = 0. 



(23) 



« By the code construction and by the packing lemma 
(Lemma O, 



limPr(5H|£o'„X = l) = 0, 

n—^0 



(24) 



whenever 

Ro + Ri,c + R2.C < HUo, U,-Yi) - 5{e). (25) 
By the code construction and by the packing lemma: 

limPr(f2»|fo»>-^ = l) = 0, (26) 

whenever 

+ R', < I{U,; Y,\Uo) - S{e). (27) 

Again, by the code construction and by the packing 
lemma: 

limPr(£:3.|f5,,-M = l)-0, (28) 

n— >0 

whenever 

Ra+Rl^c+R2,c+R^,p + R', < I{Ua,Uf,Y,)-S{e). (29) 
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Thus, we conclude that if for i G {1,2} 

R[+R'2> I(Ui;U2\Uo) + 6ie) (30a) 
R,,p + R',<I{Uf,Y,\Uo)-S{e) (30b) 
i?o + i?i,c + R2.C + R,,p + i?- < I(Uo, U^; Yi) - 5{e), (30c) 

then the average (over random codebooks, messages, and 
channel realizations) probability of error of the described 
scheme tends to as the blocklength n tends to infinity. The 
existence of a deterministic scheme with average (over mes- 
sages and channel realizations) probability of error tending to 
as n tends to infinity follows then from standard arguments. 

By the Fourier-Motzkin ehmination algorithm we conclude 
that whenever 

I{Ui;Y,\Uo) + I{U2;Y2\Uo) > I{Ui;U2\Uo) (31) 

then for every rate tuple {Rq, Ri, R2) satisfying 

Ro + Ri<I{Uo,Ui;Yi)-6{e) (32a) 
Ro + R2<I{Uo,U2;Y2)-S{e) (32b) 
Ro + Ri + R2< I{Ui-Yi\Ua) + I{U2;Y2\Ua) 

+ mill /(C/o;yO - I{Ui- U2\Uo) - S{e) 

i— 1,2 

(32c) 

2Ro + R, + R2< /(t/o, Ui;Yi) + I{Uq, f/a; Fa) 

-I{Ui-U2\Uo) - 5{e) (32d) 

for a suitable 6(e) — > as e — > 0, there exists a choice of 
the rates i?i,c, Ri.p, R2,c, R'l, R2 > such that i?i = 

Ri.p + Ri.c and i?2 = R2.P + R2.C and such that ( l30l l holds. 

Notice that for every choice of {Uo,Ui,U2, X) that does not 
satisfy ( l3Tl i we can strictly enlarge the rate region (|32] | if we 
replace the random triple {Ua, 1/1,1/2) by ([/q, f7{, J/j) where 
U[ and C/2 are constants and Uq — (Uq, 1/1,1/2). The new 
choice {Uq,U[,U2, X) moreover satisfies (l3Tl i because both 
sides are 0. Also, X can be written as a function of the new 
auxiliaries Uq, U[, 1/2- We thus conclude that the rate region 
in (I32I 1 is achievable also when dsTl i is violated. 

Taking e — > 0, now establishes the inclusion T^Marton ^ 

CnoFB- 

The following two remarks are found useful in the analysis 
of our feedback scheme in Appendix |D] 

Remark 7. Under conditions ( 131b and (I32l i there exists an 
associated choice of parameters for our scheme such that the 
associated auxiliary codewords satisfy 

Pr((f/o"(Afe), [/i"(Mc, A/i,p, LI), U^\M„ A'h.p, L*^)) 

^ '^/32(^C/oC/i!72)) 

— !► 1 as n — > 00. 

Remark 8. Inspecting the proof we see that the memoryless 
channel property has been used only to establish the limit i23i . 
The other limits ( I21l i, ( I24l i, (I26l l, and ( I28l l follow solely from 
the way we constructed the code. Suppose now we replace the 
memoryless channel with a general channel Pyn^x"- Then 
under conditions (13 11 1 and ( I32l l, there exists an associated 



choice of parameters for our scheme such that the average 
error probability goes to zero as n 00, if for i G {1, 2}: 

Pr [{U^{M,),U-{M,,A'U^„L:),Yn e %^-HPuoU.y,)) 
— > 1 as n — > 00. 

Appendix B 

Analysis of the Lossy Gray Wyner Scheme with 
Side-Information 

We analyze the failure probability Pr (f Uf'^') associ- 
ated with our random coding scheme, where f '^'^ is the event 
that Receiver i fails, i.e., (X", ¥[") ^ %"iPxv,)- 

Let ^0,0' -^1,0' -^2,0' ^1.0' -^2,0' ^0.1' ^is' -^i' ^0,2' 
K2 2, L2 be the tuple of indices chosen by the sender Also, 
let ' 

. £0 be the event that X" ^ T^/^{Px); 

> £1 be the event that 

Vfco^o, fci,o,-^i,o : 

(X", Fo"(l; fco.o, fci,o,^i,o)) ^ T^^/iiPxVo); 

• £2.1, for i G {1, 2}, be the event that 

Vfcojij ki i, £i : 

• fa.i, for i G {1, 2}, be the event that 

iV-i^■, Kl„ Kl„ LlQ),Vr{Kl^, K*,, L*),Yn 

<^t:{Pv,v,y,); 

m £4^1, for i G {1, 2}, be the event that 

+ K : 

{y^{v, kIq, kIq, LiQ),vr{Ko,„ Kl„ id, YD 

&t:{Pv,v,y,); 

m £^ i, for i G {1, 2}, be the event that 

3i,,o^LlQ,i,^L: : 

(Vo^iz; Kl„ Kl„ £,,0), Vr{Kl„Kl„ id), Y^) 

^T:{Pv,y^yd. 

Notice that whenever event {£^ n n occurs, then 
{X^^ ,Vd{Kl,^,Kl^,LX)) G T^{Pxvd- If additionally also 
event {£^.1 H j fl £^d occurs, then Receiver i produces 
V;« = l^"(^:^,„^Q;„Lf). Therefore, 

Pr(f < Pr (fo U fl U £2^^ U £3,^ U £i,, U £5^^) 
<Pr(fo)+Pr(filfo') 

+ Pr(£:2,,|£:r) + Pr(f3,«|f2',.) 

+ Pr(£:4,,)+Pr(f5,«). (33) 

We analyze each of the summands separately. Hereinafter, a 
nonnegative function (5(e) satisfying (5(e) ^ as e ^ 0, can 
be chosen such that the statements hold. 

> Since X" is Px-i-i-d. and by the weak law of large 
numbers: 

lim Pr(£:o) = 0. (34) 
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By the code construction and the covering lemma 
(Lemma O: 

lim Pr(£:i|£:n) = (35) 

n~>oo 

whenever 

R'o + Ro,o> IiX;Vo) + 5ie). (36) 
Again, by the code construction and the covering lemma; 

lim Pr(£2,.|^^f) = (37) 

whenever 

R'^ + Ro,, + R.^.,> IiV,;X,Vo) + 6{e). (38) 

The pair {Vo^it; K^,, Ll,),Vl^{K*^^, Kl,, L*)) 
depends on only through X", i.e., the Markov chain 

V^(^;K*^,,Kl,,Ly,Vr{Kl^,Kl^,L*)^X^^^r 

holds. Therefore, is Py \xVoV = JV |jf -independent 
given (X", V^^ii; K*^^, K*^, LI,), V^iK*^^, K*^*)) 
and by the conditional typicality lemma (Lemma [B: 



lim Pr{£s,,\£l^) ^0. 



(39) 



Notice that the codewords {V:^{KQ .^,K*j,£i)} for £i S 
[2"^']\{L*} are not independent and Py^-i.i.dQ How- 
ever, following similar steps as in pT] Appendix 12A], 
one can prove Inequality (HTI) on top of the next page 
for arbitrary A:*,, €*) G [2"-«"-] x [2"«'.-] x p'^-^S] 
which directly yields Inequality (021), also shown on the 
next page. Here, Equality ( l40b holds by the symmetry 
of the code construction. For (A:g /c*j, i!*) = (1,1,1), 
Inequality dTIT i is sti-aightforward; for (fcp ^, fc* J = (1, 1) 
and > 1 it follows by this first case and the symmetry 
of the code construction; and for {kQ^,k*^) ^ (1,1) 
and £* arbitrary it follows by (l43]l-(H4li which hold 
again by the symmetry of the code construction and 
because conditioned on ^ = K*^ — L* = 1 every 
set of [2"^'J - 1 codewords {Vi"iko,^,ku,e,)} for 
(fco,z, ki.i, ii) 7^ (1, 1, 1) has the same joint distribution. 
Notice that on the right-hand side of ( |42] | we have the 
probability that one of the [2"^* J independent and Py^- 

i.i.d. codewords {^^^"(1, 1, =i is jointly e-typical 
with the pair (^^"(j; if^* X*^), F,"). Thus, by the 
packing lemma (Lemma [3]l the probability on the right- 
hand side of (l42l i tends to as tends to oo whenever 



R',<I{V,;Vo,Y,)-6{e). 
We thus conclude that 

lim Pr {£4a) = 
whenever S45[ holds. 



(45) 



(46) 



'^This can be seen with the following simple example. Let the heights of two 
students Aq and Ai be uniformly distributed over the interval [1.7, 1.9] m 
and independent of each other. Also, let C be the index of the student that has 
height larger than 1.89m if this index is unique; otherwise let C be Bern(^). 
Let C be the index in {0, 1} not equal to C. Notice that Pr (Aq > 1.89) = 

whereas Pv (Ag > 1.89) = Pr (Aq > 1.89 and Ai > 1.89) = 
Thus, Aq is not uniform over [1.7, 1.9]. 



* Following similar steps as above, we can prove upper 
bound ( |47] |. Then, by the multivariate packing lemma 
(Lemma IH: 

lim Pr(£5,,) = 0, (48) 

n—^oo 

whenever 

R'o-R^M + R^<IiVo■,Y,) + I{V,■,Vo,Y,)-Sie). (49) 

Combining (l33]l with (l36]l, (l38]l, dUll, and (|49]l we obtain that 
Pr (i?'-^'') and Pr (S^^'') both tend to as n oo whenever: 

R'o + Rofi> IiX;Vo) + 5ie) (50a) 

R[ + Ro.i + Ri,i > I{Vi;X, Vo) + S{e) (50b) 

i?2 + Rq.2 + R2,2 > I{V2;X, Fo) + 6{t) (50c) 

R'a - i?i,o + < I{Vq;Yi) + I{Vi-Vq,Yi) ~ &{€) (50d) 

R'o - R2..0 + i?2 < I{Vo;Y2) + I{V2;Vo,Y2) - d{e) (50e) 

R[ < I{Vi;Vo,Y,) - 5{e) (SOT) 

R'^ < I{V2;Vo,Y2) ^ S{e). (50g) 

We now argue that with an appropriate choice of the 
auxihary rates R'q, R[, R'2, Rq.o, Rq.i, Ro,2, Ri.o^ Ri,i, 
R2,Q, R2,2 > our scheme achieves the region TZ'^q^- We 
first replace Ri i by Ri — Ri,o, for i £ {1,2} and i?o,o by 
Ro — i?o,i — Ro,2 to obtain 

R'o + Ro- Ro.i - Rq.2 > I{X- Fo) + <5(e) (51a) 
R[ + i?.o,i +Ri- Rifi > I{Vi;X, Vo) + S{e) (51b) 
R'^ + Ro,2 + R2- R2.0 > I{V2;X, Vo) + S{e) (51c) 
R'o - Ri,o + R'l < I{Vo;Yi) + I{Vi;Vo,Yi) - d{e) 

(5 Id) 

R'o - R2,o + i?2 < HVo;Y2) + IiV2;Vo,Y2) - Sie) 

(51e) 

R[<IiV,;Vo,Y,)-die) (51f) 
R2<I{V2;Vo,Y2)~S{e). (51g) 

Then, employing the Fourier-Motzkin elimination algorithm 
to eliminate the nuisance variables i?Q, R[, R^, i?o,i, ^0.2, 
-Ri,o, ^2,0, we obtain that if (i?o, ^1,-^2) satisfies 

Ro + Ri> nX; Vo) + I{Vi-X, Vo) - I{Vo; Y^) 

-I{Vi;Vo,Yi)+S{e) (52a) 

Ro + R2> I{X; Vo) + I{V2;X, Vo) - I{Vo; Y2) 

~I{V2;Vo,Y2) + 5{e) (52b) 
i?o + i?i + i?2 > I{X- Vo) + I{Vi-X, Vo) + I{V2;Xyo) 
~I{Vi-Vo,Y^)-I{V2;Vo,Y2) 
-iT,mI{Vo]Y,)+5{e) (52c) 

i 

then there exists a choice of nonnegative rates i?Q, R'l, R'2, 
Ro,i, R0.2, Ri,o, R2,Q that satisfies (ISTT i and 



Ri 


— Ri.o 


> 




— R2.2 


> 


R'o 


— Ri,o 


> 


R'o 


— R2.0 


> 


Ro.i 


— Ro,2 


> 
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Pr(£:4,,) = Pr 



y (Fo"(z; R'l,, LI,), VriKl,. Kl,,£i}, YD € T^iPv.v.Y.) 



J 



Pr I U {V^{i-J<l,,Kl,,Ll,),V:\l,\,l.),YneTl\Pv,v.Yd 

< Pr I U {V^{^■,Kl,,Kl,,Ll,).V^{l^,l^).Yn € T^{Pv.v,Y.) 
fi=i 



(40) 



Kl, = kl,,Kl, = kl,L*=i* 1 (41) 



Pr < Pr I U (l^o"(*; K*^,. Ko. ^lo), ^."(1, 1, ^0, YD G ^^."(^yoV.yJ 
£i=i 



(42) 



= Pr Ml (Fo"(»; K*,o, Ko^ ^lo), 1, ^.), S T^iPv.v.Y^ 

\ fi=2 

/L2"«;j 

< Pr U (^o"(*; ^0*0, ^;,o> ^:,o): ^^"(1, 1. ^0, i"") G T:{Pv„v.y,) 



^0,i — f^OAi^i.i — ^i.i^-^i — (43) 



^0,1 ~ '''0,1' -'^i.i ~ '^i.i' ~ 



(44) 



Pr < Pr 



y (Fo"(*;l,l,f,,o),V^,"(l,l,^.),>^,") e T^iPvoV.Y.) 

V «,e[2"«i] / 



(47) 



Due to the Markov chain (Vb, Vi, V2)^>— X^>-(yi, F2) the 
constraints in ( |52] | are equivalent to 

i?o + i?i >/(^;^o,^i|n) + '5(e) (53a) 
i?o + i?2 >/(^; 14), 1^211^2) + (5(e) (53b) 
Ro + Ri+R2> I(X; Vi\Vo,Yi) + I{X; l/zlVb, 1^2) 

+ m!ixI{X;Vo\Yi) + 6{e). (53c) 

Thus, we conclude that the region ( |53] | is e-achievable for 
all choices of the auxiliary random variable Vq satisfying the 
Markov chain (yo,Vi,V2)~o-X^>-(Yi,Y2). Letting e ^ 0, 
the achievability of T^lgw established. 

The following remark is found useful in the analysis of the 
feedback scheme in Appendix ID] 

Remark 9. In our error analysis, only Limits (I34l i and ( |39l l 

rely on the assumption that (X", ^",1^2") '^'"^ PxYiY2-l-l-d- It 
is easy to check that replacing this assumption with the more 
general assumptions 



(i) Pr(X" e TJ)^iPx)) ^ I as n ^ 00. 

{Y",Y^) is Pyj^y2\x -independent given X". 
still guarantees the existence of associated parameters such 
that the scheme above e-achieves the region ( l5iD . In particular, 

Pr {{X\ VTiK^Kl,, L*)) i T:\Pxv.)) ^ 

and 

Pr {vr^Vr{Kl,,Kl,,L*))~,0, 
for i G {1, 2}, fli n — > 00. 

Appendix C 
Convexity in Theorem[2] 

Let {Voj,Vi^j,V2,j,Xj,Yij,Y2j}je{o.i} be two sets of 
mutually independent random variables for j S {1, 2}, where 

• {Xj,Yi^j,Y2j) ^ PxYiY2'^ 

. {yo,j,v^^,,V2,j)^x,^^^^,,Y2^i). 
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. Py^^^^x, = Pv,\x for {1,2}. 

Let Q ^ Bern(a) be independent of the union of the two 
sets, and define Vq = Vo^q,V, = Vi^Q,X ^ XQ,Yi ^ Y,,q, 
for i e {1,2}. Notice that as the law of (Xi, Fi^i, l2,i) ™d 
the law of (-''^2,^1,2,^2,2) are the same, the "time-sharing" 
random variable Q is independent of the triplet (X, Yi,l2)- 
Therefore, and since by assumption 

(K),^l,V^2)— (^,Q)^(n,y2), 
~ def - 

we conclude that defining Vb = (Q, Vb) we have the Markov 
chain 

{Vo,Vi,V2)^X-^{YuY2)- (54) 
We further notice that for i G {1,2}: 

I{X-%\Vo,%,Q) ^ I{X-%\VQ,Yi) (55) 

and 

I{X-Vo\%,Q) ^ I{X-Vo,Q\%) = I{X-Vo\%), (56) 

where the first equality holds because of the independence of 
Q and {X,Yi). Moreover, by (|55]l and (ESll 

I{X-Vo,V^%,Q) = I{X-V,,Vo\%). (57) 

Combining these inequalities with the Markov condition, we 
conclude that the region 7?.™^^ is convex. 

Appendix D 
Error Analysis for the Feedback Scheme 

We bound the average probability of error (where the 
average is over the random messages, codes, and channel 
realizations). Let £ be the error event; 

2 B 

^ = U U {(A/o,.,(6),M,.(b)) 7^ (Mo.(,),M,- (,))} . 

4=1 6=1 

Moreover, for each 6 e [i? + 1], let be the error event of 
the Marton code in block h: 

2 

i=l 

Then, 

/B+l \ B 

Pr(£) < Pr U < ^Pr(J-f,|J-,%i) + Pr(J-B+i). 

\b=l I b=\ 

By construction, we have that Vy[Tb+\) — ?■ as n ^ cxd. 
Let us now analyze the probability Pr(J"b|-^b+i) for a fixed 
6 \B\. In light of Remark |8] we see that if 

J([/i;ri,Fi|(7o)+/(t/2;>"2,t^2|C/o) >/(C/i;(72|C/o); (58) 
and 

i?o + i?i </(C/o,(7i;yi,Vi)-5(e) (59a) 
i?o + i?2 </(f/o,(72; 1^2,^2) -(5(e) (59b) 
i?o + i?i + i?2 < /(t/i; v^i|c/o) + /((^2; i"2, ^2|t/o) 

+ miii/(C/o; r„ 14) - /([/i; C/2IC/0) - <5(e); 

(59c) 

2i?o + i?i + i?2 < /(c/o, (^i; ^1) + /((7o, c/2; ^2, ^^"2) 

-/([/i;[/2|[/o)-(5(e); (59d) 



and for i G {1, 2}: 

Pr((^o:w,t/;:(,),F,"(,),V;:'(,)) ^7;"(Pc/oC/.y.,yJ) ^ 

(60) 

as n ^> cxo, then there exists a choice of the parameters such 
that Pr {F^\Fl^^ ^ as n ^ 00. 

From this point forward we assume that Conditions (l58l l 
and ( |59] | hold, and prove that if additionally 

^0 + ^1 >/(C/o,(7i,C/2, ?; 1/0,^1 1^1)+ -5(6) (61a) 
i?o + i?2 >/(C/o,t/i,C/2,>^;K),V^2|i2)+<5(e) (61b) 
i?o + i?i + i?2 > /(C/o, t/i, C/2, r;Ji|^o, ^i) 
+/([/o,C/i,;72,r;F2|K),i1i) 
+ max/(f/o, C/i, (72, F; Fo|r.) + (5(e) (61c) 

i 

then the limit (l60] t holds. We notice that 

Pr(([/^^(,), V;"(,)) ^ 7;"(Pc/oC/.y.yj) 

< Pr((C/o"(6),C^/:W'^"w'Cw) ^7;"(^^oC/.y,vj) 

where ^"'^ ^^(b) denote the codewords chosen by 

the LGW-SI encoding rule \h)- We now verify that under 
conditions (1611 1. both terms on the right-hand side of (|62] | 
vanish as n ^ cxd. 

Since the input X"^^ is a component-wise function 
°f i^o{b)^^i(b)^^2{b)) '■^^ channel is memory- 

less, (yi"b),1^2"(&)'?(6)) is Py^ I j;^^^^^,^ -independent given 
(^(i\b)' "(6)' ^2 [b])- Furthermore, from Marton's code con- 
struction and in light of Remark |7] we have that under 
conditions dSSl l and (l59l l 

((t^o"(6),f^r(6),f^2"(6)) ^ '7;/32(Pc/oC/.C/j) ^ 0. 

Therefore, by the conditional typicality Lemma, also 

Pr {{Ul^,yUl^,yUl^,yY^^) i %/idPu,u.u.Y)) ^ 
as ri — > 00. 

Thus, by Remark |9] (recall we have used the parameter e/2 
for the LGW-SI code) and under conditions (I6TI 1 

p^{Kib)^yi:ib))^o (63) 

Pr(('^(l'(6),t^"(6),t^2'(f,),?(b)'^i"(6)) ^ V/2iPuoUiU2YV^)}'^^ 

(64) 

as n — 7- (X). 

Now, since ^"{b) P'y\UoUiU2Y'^^'^^P^^'^^'^^ given 
Po{b)^^i{b)^^2{b)'^{b))' Markov condition 

^i!ib) ~°~Po,{b)^UZ{b)-' ^£(&)'^(&))^^^"(6) 

holds, ( |64| | and the conditional typicality Lemma imply that 

Pr((t^^W,t/;:(„),>^rW'^"w)^'^"(^c/oC/.y.yj) ^0. (65) 

With (|62] | and ( |63] | this establishes ( l60l l. We thus proved that 
whenever ( fSSl l, ( |59] l, and ( I6TI 1 are satisfied, then the probability 
of error tends to as n — > 00, for any e small enough. 
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Employing the Fourier-Motzkin elimination algorithm on 
constraints f5% and dMT l where we replaced Ri by Ri+Ri, and 
letting e tend to 0, we obtain that under the set of constraints 
(fT2l i and when dSSb holds, then there exists a choice of the 
parameters such that the probability of error of our scheme 
tends to as n — > oo. Notice however, that when a triplet 
{Uo,Ui,U2) does not satisfy ( |58] l. then the rate region ( fT2l i 
is strictly enlarged if we replace this triplet by {Uq, U{, U2) 
where U[ and U2 are constants and Uq = (C/q, C^i, £^2)- The 
new choice (f/g, U[, U2) moreover satisfies ( fSST i because both 
sides are 0. It also satisfies the Markov chain (IZTT i and X can 
be expressed as a function of the new auxiliaries [/q, [/{, [/j- 
We can thus ignore constraint (|58] l in the statement of the 
achievable region. 

We conclude that since by (fTSl ) the effective rates of 
transmission tend to {Rq, Ri, R2) as B ^ 00, any rate triplet 
satisfying the constraints ( fT2] ) is achievable by our scheme. 
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